Universal keyboard

ABSTRACT

A biometric analyzer, including a biometric generator, receiving user input in conformance with a layout of keys, and generating therefrom time series, over a period of time, of touch location, touch timing and touch pressure data, for transmission as a data stream to a biometric identifier, wherein the time series of touch timing data includes a series of times at which keys are depressed and times at which the depressed keys are released, and wherein the time series of touch pressure data includes a series of pressure magnitudes, and a biometric identifier, receiving the time series from the biometric generator, deriving a biometric template of the user therefrom, and subsequently when the user tries to access a system using a keyboard, identifying the user based on his biometric template, and granting or restricting the user&#39;s access to the system, based on his biometric template.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 14/809,290 entitled UNIVERSAL KEYBOARD, and filed on Jul. 27,2015 by inventors Jordan A. Berger and John V. Monaco.

FIELD OF THE INVENTION

The present invention relates to electronic keyboards.

BACKGROUND OF THE INVENTION

The keyboard is one of the most universal peripheral components fordesktop and laptop computers, and yet it relies on the QWERTY systemthat dates back to the 1870's. It is arguably the most ancient part ofthe desktop and laptop computers in use today. The use of keyboards isubiquitous with word processing, web browsing, multimedia streaming andgaming.

Many applications remap keys or key sequences to application-specificcommands. For example, “Ctrl+n” creates a new document or opens a newwindow, depending on the context. The keyboard layout can be entirelyremapped through software. The standard QWERTY layout is oftenassociated with a US English key map, but many others exist. Forexample, some users switch to a DVORAK layout due to comfort, or use alanguage other than English on a QWERTY keyboard. Many applicationsallow the user to create time saving “macros”, but most users do nottake advantage of them due to the high barrier of learning how toprogram the macros. Users often purchase more than one keyboard orcontroller, each one made for a specific purpose. For word processing,some users like to use a keyboard with a relatively light touch, butwhen gaming they prefer a mechanical keyboard with a heavier pressure.Many gaming applications only utilize about 8 keys, and the unused keysbecome obsolete during gameplay. Many users in the graphics and imagingfields execute thousands of mouse clicks per day to perform tasks thatcould be highly simplified with a more intelligent human-computerinterface.

The current field of keystroke dynamics, as described in

-   -   M. Karnan, M. Akila, N. Krishnaraj, Biometric personal        authentication using keystroke dynamics: A review, Applied Soft        Computing, Vol. 11, Issue 2, March 2011, pages 1565-1573, ISSN        1568-4946, http://dx.doi.org/10.1016/j.asoc.2010.08.003,        http://www.sciencedirect.com/science/article/pil/S156849461000205X,

-   and    -   Pin Shen Teh, Andrew Beng Jin Teoh, and Shigang Yue, “A Survey        of Keystroke Dynamics Biometrics,” The Scientific World Journal,        Vol. 2013, Article ID 408280, 24 pages, 2013.        doi:10.1155/2013/408280,

-   utilizes behavioral biometric data from users, as described in    -   Fabian Monrose, Aviel D. Rubin, Keystroke dynamics as a        biometric for authentication, Future Generation Computer        Systems, Vol. 16, Issue 4, February 2000, pages 351-359, ISSN        0167-739X, http://dx.doi.org/10.1016/S0167-739X(99)00059-X,        http://www.sciencedirect.com/science/article/pil/S0167739X9900059X,

-   in order to perform a variety of important functions, such as    on-line user authentication, as described in    -   Bergadano, Francesco, Gunetti, Daniele, and Claudia Picardi,        User authentication through keystroke dynamics, ACM Transactions        on Information and System Security (TISSEC), Vol. 5, issue 4,        November 2002, pages 367-397, New York, ACM, ISSN: 1094-9224        EISSN: 1557-7406 doi:10.1145/581271.581272.

-   Researchers are studying use of keystrokes to detect physical    ailments such as arthritis and Parkinson's disease,    http://www.nature.com/srep/2015/150409/srep09678/full/srep09678.html).

Keyboards commercially available today are limited in that they can onlyprovide timing information, while it has been shown that use ofadditional sensors, such as pressure and acceleration, significantlyimproves the performance of a keystroke biometric system. The demand foradditional sensors continues to grow as keystroke dynamics isincorporated into an increasing number of applications.

Prior art virtual keyboards project onto surfaces, and will never likelybe a “preferred” keyboard for any user. Virtual keyboards have afuturistic appearance, and can be used in place of keyboards for shortsessions, but for the “normal” or “heavy” computer user, the virtualkeyboard lacks many features.

Conventional and virtual keyboards can output keystrokes (“all or none”)and timing data, but cannot measure pressure data, and lack the spatialresolution that allows, for example, estimation of finger size, limitingtheir use in advanced behavioral biometrics.

SUMMARY OF THE DESCRIPTION

Embodiments of the present invention relate to a universal keyboard,which is dynamically optimized for all key input to the computer, and isthe first fully-compatible biometric keyboard. The keyboard includes ablank translucent surface, a capacitive array touch screen thattransmits touch to the computer, and a projection system thatdynamically projects the keyboard template or layout that is needed foreach application. There are inter alia four types of projection systems:(1) micro-LED array applied to the under-surface of the keyboard, (2)projection system applied to a bar across the keyboard, (3) projectionsystem that projects onto the surface from underneath the keyboard, and(4) touchscreen system.

The universal keyboard includes inter alia a speaker, a microphone, awebcam, an accelerometer, a USB connection, and an optional wirelessmodule such as a BLUETOOTH® module.

The universal keyboard includes a device driver that initializes thekeyboard, the projection system, the microphone, the webcam and a touchpad, that initializes a BLUETOOTH® pairing, that loads a sound file, andthat dynamically projects a display file onto the keyboard. The driveralso maps touch data to ASCII keystroke or bitmap data, as appropriate,formats the keystroke or bitmap data for output, and outputs the datavia USB or such other data channel. Data output from the keyboard viathe device driver may use a file format and communications protocol thatconform to an existing or future standard.

There is thus provided in accordance with an embodiment of the presentinvention a biometric analyzer, including a biometric generator,receiving user input in conformance with a layout of keys, andgenerating therefrom time series, over a period of time, of touchlocation data, of touch timing data and of touch pressure data, fortransmission as a data stream to a biometric identifier, wherein thetime series of touch location data includes a series of locations ofkeys that are pressed by the user, wherein the time series of touchtiming data includes a series of times at which keys are depressed andtimes at which the depressed keys are released, and wherein the timeseries of touch pressure data includes a series of pressure magnitudes,and a biometric identifier, receiving the time series of touch location,touch timing and touch pressure data from the biometric generator,deriving a biometric template of the user therefrom, and subsequentlywhen the user tries to access a system using a keyboard, identifying theuser based on his biometric template, and granting or restricting theuser's access to the system, based on his biometric template.

There is additionally provided in accordance with an embodiment of thepresent invention a method for biometric analysis, including receivinguser input in conformance with a layout of keys, generating, from theuser input, time series, over a period of time, of touch location data,touch timing data and touch pressure data, wherein the time series oftouch location data includes a series of locations of keys that arepressed by the user, wherein the time series of touch timing dataincludes a series of times at which keys are depressed and times atwhich the depressed keys are released, and wherein the time series oftouch pressure data includes a series of pressure magnitudes, deriving abiometric template of the user from the times series of touch location,touch timing and touch pressure data, and subsequently, when the usertries to access a system: identifying the user based on his biometrictemplate, and granting or restricting the user's access to the system,based on the user's biometric template.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be more fully understood and appreciated fromthe following detailed description, taken in conjunction with thedrawings in which:

FIG. 1 is a simplified diagram of a keyboard for use in cooperation witha keystroke biometric analyzer, in accordance with an embodiment of thepresent invention;

FIG. 2 is a simplified diagram of a keyboard with interactive generationof layouts of keys, in accordance with an embodiment of the presentinvention;

FIG. 3 is a simplified top view of a keyboard, in accordance with anembodiment of the present invention;

FIG. 4 is a simplified side view of the keyboard of FIG. 3 showing fourlayers, in accordance with an embodiment of the present invention;

FIG. 5 is a simplified illustration of a layout of keys for wordprocessing, for use with the keyboard of FIG. 3, in accordance with anembodiment of the present invention;

FIG. 6 is a simplified illustration of a layout of keys for an alternatelanguage, for use with the keyboard of FIG. 3, in accordance with anembodiment of the present invention.

FIG. 7 is a simplified illustration of a layout of keys for aninteractive MINECRAFT® game, for use with the keyboard of FIG. 3, inaccordance with an embodiment of the present invention.

FIG. 8 is a simplified flowchart of a method for using a keyboard togenerate data for a keystroke biometric analyzer, in accordance with anembodiment of the present invention;

FIG. 9 is a simplified flowchart of a method for interactivelygenerating layouts of keys for a keyboard, in accordance with anembodiment of the present invention;

FIG. 10 is a simplified diagram of a keyboard device driver, inaccordance with an embodiment of the present invention;

FIG. 11 is a simplified diagram of a mouse device driver, in accordancewith an embodiment of the present invention;

FIG. 12 is a simplified diagram of a keyboard using a micro-LED arrayprojection system, in accordance with an embodiment of the presentinvention;

FIG. 13 is a simplified diagram of a keyboard using a projection systemapplied to a bar across the keyboard, in accordance with an embodimentof the present invention;

FIG. 14 is a simplified diagram of a keyboard using a projection systemthat projects onto the surface from underneath the keyboard, inaccordance with an embodiment of the present invention;

FIG. 15 is a simplified diagram of a keyboard using a touchscreen, inaccordance with an embodiment of the present invention; and

FIG. 16 is a simplified diagram of the biometric analyzer of FIG. 1, inaccordance with an embodiment of the present invention; and

FIG. 17 is a simplified flowchart of biometric analysis andauthentication, in accordance with an embodiment of the presentinvention.

For reference to the figures, the following index of elements and theirnumerals is provided. Similarly numbered elements represent elements ofthe same type, but they need not be identical elements.

Table of elements in the figures Element Description 100 keyboard 110blank transparent surface 120 capacitive surface 140 projection system150 controller 160 speaker 165 microphone 170 webcam 175 accelerometer180 USB connector 185 wireless module 190 biometric generator 200keyboard 210 blank transparent surface 220 capacitive surface 240projection system 250 controller 260 speaker 265 microphone 270 webcam275 accelerometer 280 USB connector 285 wireless module 290 dynamickeyboard layout generator 300 keyboard 310 finished acrylic material 320alloy bond metal cover 330 microprocessor 340 lithium ion battery 350micro-USB charging port 360 LED 410 silicone layer 420 touch sensorlayer 430 acrylic layer 440 LED layer 450 acrylic blocks 500 layout ofkeys 510 character keys 520 space bar 530 cancel key 540 specialcharacter key 550 copy key 560 paste key 570 touch pad 580 sensitivityscroll bar 600 layout of keys 610 character keys 620 space bar 630special keys 640 keys for language selection 650 key for adding alanguage 680 sensitivity scroll bar 700 layout of keys 710 directionalkeys 720 special key 730 special key 740 special key 750 special key 800method 810 flowchart operation 820 flowchart operation 830 flowchartoperation 900 method 910 flowchart operation 920 flowchart operation 930flowchart operation 1000 keyboard driver 1100 mouse driver 1200 keyboardembodiment using micro-LED array projection 1210 silicone layer 1220capacitive layer 1230 acrylic layer 1240 micro LED layer 1300 keyboardembodiment using projection bar 1310 acrylic keyboard 1340 projectionbar 1400 keyboard embodiment using projection underneath keyboard 1410acrylic layer 1420 support layer 1440 projection device 1500 keyboardembodiment using touchscreen 1510 touchscreen 1600 biometric analyzer1610 biometric identifier 1620 biometric authenticator 1630 biometricbehavioral analyzer 1640 biometric learning machine 1700 method 1710flowchart operation 1720 flowchart operation 1730 flowchart operation1740 flowchart operation 1750 flowchart operation 1760 flowchartoperation 1770 flowchart operation 1780 flowchart operation 1790flowchart operation 1800 obfuscator module

DETAILED DESCRIPTION

Embodiments of the present invention relate to a universal keyboard,referred to herein as the “ONE-KEYBOARD”, which is a universal solutionto all key input to the computer, and is the first fully-compatiblebiometric keyboard.

The keyboard consists of a blank translucent surface, a capacitive arraytouch screen that transmits touch to the computer, and a projectionsystem that projects the keyboard template or layout that is needed foreach application. As described below, there are inter alia four types ofprojection systems: (1) micro-LED array applied to the under-surface ofthe keyboard, (2) projection system applied to a bar across thekeyboard, (3) projection system that projects onto the surface fromunderneath the keyboard, and (4) touchscreen system.

The ONE KEYBOARD comes in a variety of sizes and in a software-onlyversion. A user has a choice of silicone pads that adhere to the surfaceof the ONE KEYBOARD, in order to find the preferred “touch”. The ONEKEYBOARD is supported on two legs that are adjustable to the user'spreferred angle. A small speaker in the keyboard allows the user tochoose from a variety of pre-recorded sounds to simulate the “click” ofa key and provide minimal haptic feedback. The user can choose a binarysound, which makes a sound at a single decibel level, or variable soundsthat are louder or softer depending on the pressure applied to the key.The ONE KEYBOARD comes in wired or wireless (e.g., BLUETOOTH®) models.Both models display the internal circuitry through the acrylic, foraesthetic purposes. The touch sensors may be set to whatever thresholdthe user prefers, and the threshold may vary based on the applicationbeing used. There are optional accessories, including inter alia amouse, microphone, webcam and speaker.

Reference is made to FIG. 1, which is a simplified diagram of a keyboard100 for use in cooperation with a keystroke biometric analyzer, inaccordance with an embodiment of the present invention. As shown in FIG.1, keyboard 100 includes a blank translucent surface 110 for use as aninput device. Translucent surface 110 may be inter alia a siliconesurface. A capacitive layer 120 is mounted underneath translucentsurface 110, for enabling detection of touch location and touch pressureon translucent surface 110. A projection system 140 projects a visuallayout of keys of a keyboard on translucent surface 100. A controller150 includes circuitry to control operation of the components ofkeyboard 100. Keyboard 100 includes a speaker 160, a microphone 165, awebcam 170, an accelerometer 175, and a USB connector 180. Accelerometer175 measures small movements in the keyboard induced by a user'sactions. Keyboard 100 also includes an optional wireless module 185, forshort-range wireless communication such as BLUETOOTH®.

Keyboard 100 includes a biometric generator 190 operative to receiveuser input in conformance with the projected layout of keys, and togenerate therefrom a time series of touch location and touch pressuredata, for use as data by a keystroke biometric analyzer 1600. Biometricanalyzer 1600 is described below with reference to FIG. 16. Keyboard 100also includes an obfuscation module for protecting a user's privacy.Obfuscation module 1800 is described below with reference to FIG. 17.

Reference is made to FIG. 2, which is a simplified diagram of a keyboard200 with interactive generation of layouts of keys, in accordance withan embodiment of the present invention. As shown in FIG. 2, keyboard 200includes a blank translucent surface 210 for use as an input device.Translucent surface 210 may be inter alia a silicone surface. Acapacitive layer 220 is mounted underneath translucent surface 210, forenabling detection of touch location on translucent surface 210. Aprojection system 240 dynamically projects a plurality of visual layoutsof keys of a keypad on translucent surface 210, where each visual layoutincludes ASCII character keys or graphical buttons. A controller 250includes circuitry to control operation of the components of keyboard200. Keyboard 200 includes a speaker 260, a microphone 265, a webcam270, an accelerometer 275, and a USB connector 280. Accelerometer 275measures small movements in the keyboard induced by a user's actions.Keyboard 200 also includes an optional wireless module 285, forshort-range wireless communication such as BLUETOOTH®.

Keyboard 200 includes a dynamic keyboard layout generator 290 operativeto dynamically control projection system 240 to project differentlayouts of keys on translucent surface 210 in response to user activityon a computing device, to receive user input in conformance with acurrently projected layout of keys, and to generate therefrom a timeseries of ASCII characters or button selections for input to thecomputing device.

It will be appreciated by those skilled in the art that the embodimentsshown in FIGS. 1 and 2 may be combined into an embodiment that combinesbiometric generator 190 with dynamic keyboard layout generator 290.

Reference is made to FIG. 3, which is a simplified top view of akeyboard 300, in accordance with an embodiment of the present invention.As shown in FIG. 3, keyboard 300 is approximately 13″ in length and 5.5″in width, and is formed by an acrylic or other translucent material 310including inter alia glass, plexi-glass and a combination of suchmaterials. The components of keyboard 300 are covered by an alloy bondmetal cover 320 having a width of 0.75″, with upper corners curved in anarc of a circle of 0.5″ radius. Keyboard 300 includes a microprocessor330 for registering and sending key information to a database, an 1100mAh lithium ion battery 340, a micro-USB connector charging port 350,and an LED 360.

Although element 310 is indicated as being an acrylic material, this isnot necessary for practice of the invention, and element 310 mayalternatively be comprised of glass, plexi-glass or such othertranslucent material, or a combination of such materials.

Reference is made to FIG. 4, which is a simplified side view of keyboard300 showing four layers, in accordance with an embodiment of the presentinvention. As shown in FIG. 4, keyboard 300 includes an upper layer 410of silicone having a thickness of 0.05″, to provide user feedback.Beneath layer 410 is a layer 420 having a thickness of 0.15″ with atouch sensor. Beneath layer 420 is a layer 430 of acrylic having athickness of 0.5″, to provide sturdiness. Beneath layer 430 is a layer440 having a thickness of 0.05″, with a programmable high resolution LEDscreen. Underneath keyboard 300 are acrylic blocks 450, to angle thekeyboard for ease of use.

Reference is made to FIG. 5, which is a simplified illustration of alayout of keys 500 for word processing, for use with keyboard 300, inaccordance with an embodiment of the present invention. Layout 500includes QWERTY character keys 510, a space bar 520, a cancel key 530, aspecial character key 540, and respective copy and paste keys 550 and560. Layout 500 also includes a touch pad 570, and a scroll bar 580 fortouch sensitivity.

Reference is made to FIG. 6, which is a simplified illustration of alayout of keys 600 for an alternate language, for use with keyboard 300,in accordance with an embodiment of the present invention. Layout 600includes character keys 610 for a Korean alphabet, a space bar 620,special keys 630, and keys 640 for language selection. A key 650 isprovided for adding additional languages. Layout 600 also includes ascroll bar 680 for touch sensitivity.

Reference is made to FIG. 7, which is a simplified illustration of alayout of keys 700 for an interactive MINECRAFT® game, manufactured byMojang Synergies AB of Stockholm, Sweden, for use with keyboard 300, inaccordance with an embodiment of the present invention. Layout 700includes directional keys 710, and respective keys 720, 730, 740 and 750for “FLY”, “TALK”, “INVENTORY” and “SPRINT”.

It will be appreciated by those skilled in the art that the layouts 500,600 and 700 of respective FIGS. 5, 6 and 7, are interactively changedvia projection system 240. In particular, both the appearance and thefunction of the keyboard layout changes dynamically, based on operationsperformed by a user.

Reference is made to FIG. 8, which is a simplified flowchart of a method800 for using a keyboard to generate data for a keystroke biometricanalyzer, in accordance with an embodiment of the present invention. Atoperation 810, projection system 140 (FIG. 1) projects a visual layoutof keys onto translucent surface 110. At operation 820, capacitive layer120 dynamically senses touch location and touch pressure on translucentsurface 110 at a sequence of times. At operation 830, biometricgenerator 190 generates a data stream of touch location and touchpressure data for use by a keystroke biometric analyzer.

Reference is made to FIG. 9, which is a simplified diagram of a method900 for interactively generating layouts of keys for a keyboard, inaccordance with an embodiment of the present invention. At operation910, dynamic keyboard layout generator 290 (FIG. 2) dynamically controlsprojection system 240 to project different ones of a plurality of visuallayouts of keys of a keypad on translucent surface 210, in response touser activity on a computing device, where each visual layout comprisesASCII character keys or graphical buttons. At operation 920, capacitivelayer 220 dynamically senses touch locations on the translucent surfaceat a sequence of times. At operation 930, dynamic keyboard generator 290generates a data stream, such as a data stream of ASCII characters orbutton selections, at a sequence of times, for input to the computingdevice, based on the projected layout of keys and the sensed touchlocations, at each time in the sequence.

It will be appreciated by those skilled in the art that the methodsshown in FIGS. 8 and 9 may be combined into a method that combinesbiometric data generation with dynamic keyboard layout generation.

Reference is made to FIG. 10, which is a simplified diagram of akeyboard device driver 1000 for the ONE KEYBOARD, in accordance with anembodiment of the present invention. As shown in FIG. 10, functions ofthe keyboard driver include inter alia:

-   -   A. initializing the keyboard;    -   B. locking the computer, in response to any attempt to disable        the driver;    -   C. initializing a speaker and loading a sound file;    -   D. initializing a microphone;    -   E. initializing a BLUETOOTH® pairing;    -   F. initializing a webcam;    -   G. initializing a projection system;    -   H. projecting a display file onto the keyboard;    -   I. initializing a touch pad;    -   J. obtaining a time series of touch data from the touch pad; and    -   K. formatting the touch data for output as a continuous data        stream (X, Y, T_(D), T_(R), P, A), where        -   X is the horizontal location,        -   Y is the vertical location,        -   T_(D) is the time (in milliseconds) that the key is            depressed,        -   T_(R) is the time (in milliseconds) that the key is            released,        -   P is the pressure (in milligrams) placed upon the key, and        -   A is the acceleration vector (in m/s²) for small movements            in the keyboard induced by a user's actions;    -   L. map touch data to ASCII keystroke or bitmap data, as        appropriate;    -   M. format the keystroke or bitmap data for output; and    -   N. output via USB.

Data output from the keyboard via a device driver may use a file formatand communications protocol that conform to an existing or futurestandard. The lowest level output from the ONE KEYBOARD is thecontinuous data stream of data (X, Y, T_(D), T_(R), P, A). In addition,the driver estimates the user's finger size, S, using an edge-detectionalgorithm, where S is the estimated two-dimensional area of the user'sfinger as estimated by detecting the diameter, d, of the finger whilethe key is depressed (e.g., S=¼πd²). The raw pixels covered by thefinger are also made available.

When appropriate, touch data is converted (i.e., mapped) onto ASCIIkeystrokes, and, when needed, the data is converted to graphical data.For example, if the user presses the keyboard where the “J” key islocated, the ASCII output for 3 is sent with the associated pressuremeasurement; if the user creates a signature on a touchpad, thesignature is mapped to a bitmap file, with a corresponding “userpressure matrix”, which is a 3D matrix containing the 2D pressureapplied along a time axis.

Keyboard device driver 1000 may be implemented in software, firmware,hardware, or a combination of software, firmware and hardware.

Reference is made to FIG. 11, which is a simplified diagram of a mousedevice driver 1100, in accordance with an embodiment of the presentinvention. The mouse that accompanies the ONE KEYBOARD is essentially aminiature version of the keyboard, and is incorporated into the sameapplication. Many optical mouse devices have been created with more thanone button or wheel, such as the gaming mice(http://www.razerzone.com/gaming-mice) manufactured by Razer Pte Ltd ofSingapore, and such as the MAGIC MOUSE®(http://www.apple.com/magicmouse/) manufactured by Apple Inc. ofCupertino, Calif., in order to facilitate the user's interaction withprograms that require the same operations over and over again. This ispopular in the gaming community. As another example, a radiologist usesthe same features of “Zoom”, “Pan”, “Window” and “Level” to interpret 50or 100 medical images a day. At this time, he can either use the samerepetitive mouse clicks, or try to find a mouse with some added buttons;the mouse that accompanies the ONE KEYBOARD creates a custom panel ofbuttons. For anyone who does work that requires repetitive tasks, themouse that accompanies the ONE KEYBOARD is a highly ergonomic solution.The mouse displays user-definable buttons, and collects a small subsetof biometric data.

Mouse device driver 1100 may be implemented in software, firmware,hardware, or a combination of software, firmware and hardware.

The ONE KEYBOARD employs a projection system to dynamically adapt alayout of keys to the user's application. If the user is typing adocument, the projected layout of keys conforms to a standard keyboard,and switches between languages, mathematical symbols and graphics, asneeded. For a user who uses more than one language, the projected layoutof keys includes keys in any language, and further includes a“Translate” button that enables the user to type in one language andhave it translated to another language. There are hundreds of keyboardlayouts being used throughout the world today, any of which may beprojected on the ONE KEYBOARD, and the projected keys may be a singlecolor, or may be color-coded, or may be any other design. When the useris working on a photo-book, for example, with a website such asSHUTTERFLY®, owned by Shutterfly, Inc. of Redwood City, Calif., the ONEKEYBOARD projects a section that shows inter alia a “Page Layout”button, a “Background” button, an “Add-a-Page” button, a “Theme” button,and a “Color” button. When the user adds photos to the book, the ONEKEYBOARD projects a section that shows inter alia an “Add Photos fromComputer” button, an “Add Photos from Shutterfly” button, and an “AddPhotos from Instagram” button. There are icons of photos, text and otherobjects that the user may drag into his book. The user may use gesturesto edit a photo, resize the photo, change the contrast, brightness, hue,saturation, and make other adjustments. When the user switches betweenapplications, such as working on a document, and then opening anInternet browser to look something up for the document, the keyboardswitches between modes optimized for each application, and produces aset of custom buttons such as “Copy”, “Paste” and “Create Hyperlink”, tofacilitate the interaction between applications. The keyboard works in ahighly synchronized fashion with the user, creating the correct keys andicons for each application, and eliminating the need for hundreds ofmouse clicks. If authentication is needed, the collected biometric datais used to verify the identity of the user using an external biometricanalyzer.

As described below, there are inter alia four embodiments of projectionsystems: (1) micro-LED array applied to the under-surface of thekeyboard, (2) projection system applied to a bar across the keyboard,(3) projection system that projects onto the surface from underneath thekeyboard, and (4) touchscreen system.

Reference is made to FIG. 12, which is a simplified of a keyboard 1200using a micro-LED array projection system, similar to keyboard 300 shownin FIG. 4, in accordance with a first embodiment of the presentinvention. Shown in FIG. 12 is a silicone surface 1210, exposed fortouch by a user. Underneath silicone surface 1210 is a capacitive touchlayer 1220, for detecting touch location and touch pressure when a usertouches silicone surface 1210. Underneath capacitive touch layer 1220 isan acrylic layer 1230.

A pattern of keys is projected onto silicone surface 1210 by a micro LEDarray 1240, underneath acrylic layer 1230.

A controller (not shown) receives user input in conformance with theprojected layout of keys, and generates a time series of touch locationand touch pressure data therefrom. The touch location and pressure datamay be used inter alia by a keystroke biometric analyzer, as explainedbelow.

Reference is made to FIG. 13, which is a simplified diagram of akeyboard 1300 using a projection system applied to a bar across thekeyboard, in accordance with an embodiment of the present invention.Shown in FIG. 13 is an acrylic keyboard 1310 with a projection bar 1340.

Reference is made to FIG. 14, which is a simplified diagram of akeyboard 1400 using a projection system that projects onto the surfacefrom underneath the keyboard, in accordance with an embodiment of thepresent invention. Shown in FIG. 14 is an acrylic surface 1410. Asupport layer 1420 is underneath acrylic surface 1410. Within supportlayer 1420 is a projector device 1440, which projects a pattern of keysonto acrylic surface 1410.

Reference is made to FIG. 15, which is a simplified diagram of akeyboard 1500 using a touchscreen, in accordance with an embodiment ofthe present invention. Keyboard 1500 uses a touch pad 1510.

Two or more of the projection systems of FIGS. 12-15 may be combinedtogether into a multi-projection keyboard.

One of the central features of the ONE KEYBOARD is that it usesbehavioral biometrics to learn the touch patterns of every individual,via a process termed “keystroke dynamics”. This is a security feature tosupplement conventional means of authentication, such as username andpassword, and may also be used as a form of error correction. At thebasic level, the device registers the behavioral data associated witheach time the user touches the keyboard. Over a short period of time,the data gathered creates a “behavioral profile” for the user. Thebehavioral profile is the set of numeric and categorical features thatdescribe the keyboard usage history, including key press and releasetimings, pressure, acoustics, keyboard motion, and the shape of thefinger mapped onto a two-dimensional space. From the behavioral profile,biometric information can be extracted to create a “biometric template”.The biometric template is a reduced set of features that are highlyreproducible and specific for each individual. E.g., the pressure andfinger shape may be used to uniquely identify a particular user withhigh probability. Some of the variations in pressure and finger shapebetween different users can be attributed to (a) physical traits of theuser, such as finger size and strength, (b) the distance from the centerof the keyboard, and (c) the user's typing proficiency. Once created,the template is a valuable part of the system. The biometric templatemay be used to grant and restrict access to a system, identify thekeyboard owner, and generate a unique encryption key that can be used todigitally sign documents.

Reference is made to FIG. 16, which is a simplified diagram of biometricanalyzer 1600, in accordance with an embodiment of the presentinvention. As shown in FIG. 16, biometric analyzer 1600 includes fourprimary components; a biometric identifier 1610, a biometricauthenticator 1620, a biometric behavioral analyzer 1630, and abiometric learning machine 1640.

Biometric identifier 1610 generates and stores the user's biometrictemplate, which represents the unique and identifiable behaviorattributes of the user. As the user's behavior changes over time, suchas due to increased typing proficiency, typing impairments, and changesin physiology, the biometric template is updated. This is necessary soas to mitigate the “template aging effect”, a phenomenon encountered inbiometrics in which the user's biometric template becomes less effectiveover time. Biometric learning machine 1640 implements an online learningmechanism to adapt to these changes in the user's behavior and torespond robustly to changes in the environment, such as keyboardpositioning and ambient noises or vibrations.

Biometric authenticator 1620 operates in two modes: static andcontinuous. In static mode, an authentication or identification decisionis made at discrete points in time using all available information up tothat point, such as at the beginning of a session or logging into awebsite. In continuous mode, an authentication or identificationdecision is made continuously as the user interacts with the keyboard;for authentication decisions in this mode, the biometric authenticatorchooses to either allow the session to continue, deeming the user asgenuine, or blocks the user from the session, deeming the user as animpostor. For identification decisions in this mode, the keyboardcontinuously recognizes the identity of the active user, such as in ashared multi-user environment.

“Affective computing” is a field of study that aims to build systemscapable of detecting and responding to the user's affect, or emotionalstate. In a desktop or laptop environment, the ability to detect theuser's affective state can enable a more integrated and productiveenvironment. Biometric behavioral analyzer 1630 recognizes the affectivestate of the user from the recorded behavior profile in order to providea more robust and dependable computing environment. Affective states andpossible responses include inter alia:

Affective State Response stress dynamically adjusting the keyboardinterface to decrease user workload and increase productivityfrustration dynamically optimizing the keyboard interface to reduceerrors; and confusion dynamically adjusting the keyboard interface toprovide assistance and additional relevant information

With the user's behavioral profile, the ONE KEYBOARD improves workflow.Based on patterns gleaned from the user's experience, the ONE KEYBOARDcorrects common mistakes made repeatedly by the user, and suggestsmoving the position or layout of various aspects of the keyboard forimproved comfort. By determining the size of a user's fingers, and thetype and number of errors made by the user, the ONE KEYBOARD suggestschanges in the layout of keys that can improve the user's experience.E.g., a larger set of keys may be more efficient for certain users. On awider scale, a company may utilize aggregated behavioral profile data toidentify patterns among large numbers of employees that might be slowingproductivity. A cloud-based system, when applied to the user profiledata, determines ways to improve workflow in a widely used program, forexample, such as PHOTOSHOP®, developed and marketed by Adobe Systems ofSan Jose, Calif. Software developers may desire the ability to studyaggregated behavioral data in order to improve the development of theirnext generation of applications.

Security is another major component of the ONE KEYBOARD. With the user'sbiometric template, the ONE KEYBOARD quickly detects when someone otherthan an authorized user is trying to access the system. Within a fewlines of typing, the biometric template of the typist discriminatesbetween an authorized user and an intruder. Companies interested innetwork security use this as a means of ensuring that only the correctuser accesses each device. Common examples of this are on-line coursesand test-taking, on-line e-commerce, and social networking companies whowish to prevent on-line bullying by “anonymous” users. Once the ONEKEYBOARD is attached to a computer, the driver can prevent anyone fromdetaching it and attempting to access the computer with a non-biometrickeyboard. Currently, many behavioral biometric programs not only rejectintruders, but they actually identify the intruder by their ownbehavioral biometric template.

Protecting the user's privacy is another major feature of the ONEKEYBOARD. Keystroke dynamics is a technique that can be used tolegitimately identify and authenticate a user by a trusted application,such as when logging into a secure banking website or during an onlinecourse. However, there are numerous scenarios in which a user'skeystroke dynamics are exposed to an untrusted application. This dilemmais often encountered in web applications, whereby a single webpage mayload dozens of third party modules that provide functionality through anexternal application programming interface (API). Given the lack ofspecial permissions required to capture keyboard events in modern webbrowsers, an untrusted web application or third-party module canpassively record the user's key press and release timings and use thisinformation to track the user's identity. This presents a privacyconcern since a malicious application can perform user identificationand verification remotely via keystroke dynamics without the user'scooperation or knowledge. Since this type of attack relies only on theuser's typing behavior, the user's identity may be compromised even whenaccessing the web application through an anonymizing network, such asThe Onion Router (TOR). From this perspective, keystroke dynamicsrepresents a form of “behavioral tracking”, the process by which anadvertiser or other third party is able to track user identity anddemographics based on his online activity.

Obfuscation module 1800 of The ONE KEYBOARD mitigates this threat bymasking the user's keystroke dynamics from any application whichreceives keyboard input. Obfuscation module 1800 introduces a smallrandom delay to each key press, key release, and touch event bytemporarily buffering the event on the device before releasing it to thehost computer. The buffer duration is chosen in such a way so as to meettwo criteria: 1) the user's keystroke dynamics appear obfuscated to anyapplication that receives keyboard input, mitigating the possibility ofan untrusted application from performing user identification orverification; and 2) so as to maximize the responsiveness of thekeyboard, introducing a delay that is unnoticeable to the user. The ONEKEYBOARD is the first keyboard designed to protect the user's privacy inreal time with a random delay that adapts to the user's typing speed.

There are numerous legitimate uses of keystroke dynamics employed bytrusted applications and the ONE KEYBOARD may preserve the intendedfunctionality of these applications. Such behavioral biometric servicesare provided by companies including TypingDNA, Behaviosec, KeyTrac. TheONE KEYBOARD is compatible with all of these applications, granted theyare trusted by the user. This functionality is provided through anapplication-specific permissions mechanism, whereby the user may chooseto trust certain applications, granting them access to the user'sun-obfuscated keystroke timings, while allowing other untrustedapplications access only to the obfuscated keystroke timings.

Using the ONE KEYBOARD and its associated methodology, on-line learningsites such as Coursera of Mountain View, Calif., and Khan Academy of NewYork, N.Y., testing companies such as The College Board of New York,N.Y., and ACT of Iowa City, Iowa, and any company seeking toverify/authenticate users who are accessing their systems via a remoteconnection, will increase the security of their systems dramatically.

Reference is made to FIG. 17, which is a simplified flowchart ofbiometric analysis and authentication, in accordance with an embodimentof the present invention. At operation 1710, biometric identifier 1610generates a behavioral profile and a biometric template of the user. Atoperation 1720 the user attempts to access a system using keyboard 100.At operation 1730 biometric authenticator 1620 identifies the user basedon his biometric template. At operation 1740 biometric authenticator1620 grants or restricts the user's access to the system, based on theuser's biometric template.

At operation 1750 obfuscation module 1800 obfuscates the user's keypress and release timings, to prevent suspicious applications fromrecording the user's key press and release timings and tracking theuser's identity. Obfuscation of key press and release timings may beperformed inter alia by temporarily buffering the key press and releaseevents, thereby introducing buffer duration errors into the timings.

At operation 1760 biometric authenticator 1620 grants trustedapplications access to the user's un-obfuscated key press and releasetimings. At operation 1770 biometric authenticator 1620 makes continuousdecisions as to the identity and authenticity of the user, based on hisbiometric template. At operation 1780 biometric behavioral analyzer 1630responds to the user's affective state by instructing projection system140 to update the layout of keyboard 100. At operation 1790 behaviorallearning machine 1640 adaptively updates the user's biometric templateto compensate for template aging and changes in the environment.

An important component of the ONE KEYBOARD is software device driver1000 for the keyboard, shown in FIG. 10, which contains the necessarycalibration and settings for the keyboard to be customized for eachuser.

Biometric learning machine 1640 uses a biometric learning algorithm.This algorithm collects data from the keyboard and then utilizes thatdata to “learn” from each user's experiences. Typing mistakes tend to berepetitive, such as touching a certain key too lightly, or misspellingsome specific words because of inverting the letters. If a usermisspells a word repeatedly, the algorithm determines if the error isdue to incomplete activation of a key, or due to another error such asinversion of letters. It then maintains a file of these learnedexperiences for each user and compensates for them, so that the userexperiences an error-free interaction. Preferably, the learningalgorithm is separate from the ONE KEYBOARD. At present, there arenumerous commercial entities utilizing biometric data. The ONE KEYBOARDis compatible with all of these applications.

Over time, biometric learning machine 1640 determines which applicationsa user uses most of the time. The universal keyboard suggests optimalkeyboard layouts, based on the applications used most of the time, whichenable a user to decrease his number of keystrokes, and improve hisefficiency and experience.

The ONE KEYBOARD comes with a device driver. In addition, there is asmall program that allows the user to choose from standard keyboardlayouts, or design his own custom layout, using a simple graphicalinterface. There is an error-correcting program that corrects typingerrors, similar to SWIFTKEY®, developed and manufactured by TouchTypeLimited of London, UK. There is an optional cloud based service thatincludes better “learning” from the user's experiences, and securitysystems that ensure that each user matches their biometric securityprofile.

The ONE KEYBOARD is the most innovative change to human-computerinteraction (HCI,http://en.wikipedia.org/wiki/Human%E2%80%93computer_interaction) withdesktop and laptop computers in the past decade, and is the lastkeyboard anyone will ever need to buy.

One having the benefit of the subject disclosure will appreciate thatthere are many variations of the keyboard of the subject invention. Thepresent invention may be embodied in applications for cellular phones,including inter alia the IPHONE® and IPAD® manufactured by AppleCorporation of Cupertino, Calif., and the ANDROID™ phones manufacturedby Samsung Electronics Co., Ltd of Korea, using built-in technology ofthe phones to collect biometric data.

Furthermore, add-on components to the ONE KEYBOARD device driver makeuse of the behavioral data collected during operation. These componentsinter alia detect fatigue and stress, detect mental states and/or moods,and diagnose physical ailments such as arthritis and Parkinson'sdisease. As such, the ONE KEYBOARD may be used by qualified medicalprofessionals. Alternatively, or additionally, such information may beused to determine when a person may be more likely persuaded by aparticular type of advertisement.

In the foregoing specification, the invention has been described withreference to specific exemplary embodiments thereof. It will, however,be evident that various modifications and changes may be made to thespecific exemplary embodiments without departing from the broader spiritand scope of the invention as set forth in the appended claims.Accordingly, the specification and drawings are to be regarded in anillustrative rather than a restrictive sense.

What is claimed is:
 1. A biometric analyzer, comprising: circuitry for abiometric generator, receiving user input in conformance with a layoutof keys, and generating therefrom time series, over a period of time, oftouch location data, of touch timing data and of touch pressure data,for transmission as a data stream to a biometric identifier, wherein thetime series of touch location data comprises a series of locations ofkeys that are pressed by the user, wherein the time series of touchtiming data comprises a series of times at which keys are depressed andtimes at which the depressed keys are released, and wherein the timeseries of touch pressure data comprises a series of pressure magnitudes;and circuitry for a biometric identifier, receiving the time series oftouch location, touch timing and touch pressure data from said biometricgenerator, deriving a biometric template of the user therefrom, andsubsequently when the user tries to access a system using a keyboard,identifying the user based on his biometric template, and granting orrestricting the user's access to the system, based on his biometrictemplate; and circuitry for an obfuscator module obfuscating the user'skeystroke timings by introducing a random time delay to each key pressand release event by temporarily buffering the event, and therebypreventing a suspicious application from performing user identificationand verification based on the user's keystroke dynamics.
 2. Thebiometric analyzer of claim 1, wherein said biometric generator furthergenerates a time series of acoustic data comprising a raw acousticwaveform recorded during keyboard operation, and wherein said biometricidentifier derives the biometric template of the user also from the timeseries of acoustic data.
 3. The biometric analyzer of claim 1, whereinsaid biometric generator further generates a time series of motion datacomprising a raw accelerometer and gyroscopic waveform recorded duringkeyboard operation, and wherein said biometric identifier derives thebiometric template of the user also from the time series of motion data.4. The biometric analyzer of claim 1, further comprising circuitry for acontinuous biometric authenticator, receiving in real-time the timeseries of touch location, touch timing and touch pressure data generatedby said biometric generator, and making continuous decisions as to theidentity and authenticity of the user based on the user's biometrictemplate.
 5. The biometric analyzer of clam 4, further comprising: aprojection system dynamically projecting a visual layout of keys; andcircuitry for a biometric behavioral analyzer, coupled with saidcontinuous biometric authenticator, recognizing intentions and affectivestates of the identified user based on the time series of touchlocation, touch timing and touch pressure data, and responding to user'saffective states by instructing said projection system to update thelayout of the keyboard.
 6. The biometric analyzer of claim 4, furthercomprising circuitry for a biometric learning machine, adaptivelyupdating the user's biometric template over time to compensate for theeffects of template aging and changes in the user's surroundingenvironment.
 7. The biometric analyzer of claim 1, wherein saidobfuscator module grants a trusted application permission to capture theuser's keystroke dynamics by granting the trusted application access tothe un-obfuscated keystroke timings.
 8. A method for biometric analysis,comprising: receiving user input in conformance with a layout of keys;generating, from the user input, time series, over a period of time, oftouch location data, touch timing data and touch pressure data, whereinthe time series of touch location data comprises a series of locationsof keys that are touched by the user, wherein the time series of touchtiming data comprises a series of times at which keys are depressed andtimes at which the depressed keys are released, and wherein the timeseries of touch pressure data comprises a series of pressure magnitudes;deriving a biometric template of the user from the times series of touchlocation, touch timing and touch pressure data; and subsequently, whenthe user tries to access a system: identifying the user based on hisbiometric template; and granting or restricting the user's access to thesystem, based on the user's biometric template, obfuscating the user'skeystroke timings comprising introducing a random time delay to each keypress and release event by temporarily buffering the event, and therebypreventing a suspicious application from performing user identificationand verification based on the user's keystroke dynamics; and granting atrusted application permission to capture the user's keystroke dynamicscomprising granting the trusted application access to the un-obfuscatedkeystroke timings.
 9. The method of claim 8, wherein said generatingfurther generates a time series of acoustic data comprising a rawacoustic waveform recorded during keyboard operation, and wherein saidderiving derives the biometric template of the user also from the timeseries of acoustic data.
 10. The method of claim 8, wherein saidgenerating further generates a time series of motion data comprising araw accelerometer and gyroscopic waveform recorded during keyboardoperation, and wherein said deriving derives the biometric template ofthe user also from the time series of motion data.
 11. The method ofclaim 8, further comprising: receiving the time series of touchlocation, touch timing and touch pressure data in real-time; and makingcontinuous decisions as to the identity and authenticity of the userbased on the user's biometric template.
 12. The method of claim 11further comprising: recognizing intentions and affected states of theidentified user based on the time series of touch location, touch timingand touch pressure data; and responding to the user's affective statesby updating the layout of the keyboard.
 13. The method of claim 11further comprising adaptively updating the user's biometric templateover time to compensate for the effects of template aging and changes inthe user's surrounding environment.
 14. The method of claim 8 whereinthe buffer durations are chosen so as (i) to obfuscate the user'skeystroke dynamics and thereby prevent an application from performinguser identification and verification based on the user's keystrokedynamics, and (ii) to maximize responsiveness of the application.